feat: SLS-494 run users' worker initialization code - #567
feat: SLS-494 run users' worker initialization code#567jasonwang-runpod wants to merge 6 commits into
Conversation
Add optional `initializer` and `init_timeout` to the serverless config. The worker runs the initializer (e.g. model load / engine start) before it starts taking jobs. On failure or timeout it emits a structured `init_failed` log line, best-effort reports the reason to the platform, and exits non-zero so the existing backoff respawns it — instead of a broken worker sitting silently IN_QUEUE until the request times out. Opt-in and a no-op when no initializer is configured.
Sync and async initializers, InitializerError wrapping, init_timeout via SIGALRM, loop-gating (the job loop starts only after init succeeds), the init_failed payload shape, and the best-effort POST URL derivation.
There was a problem hiding this comment.
Pull request overview
This PR introduces a supervised startup phase for serverless workers by adding an optional initializer callable and init_timeout to the serverless worker config, ensuring model/engine startup failures (or hangs) are surfaced quickly and the worker exits before taking jobs.
Changes:
- Added
rp_initializermodule to run an initializer with timeout handling, structuredinit_failedlogging, and best-effort platform reporting. - Updated the worker startup sequence to run initialization before starting the job loop.
- Added unit tests covering sync/async initializers, error wrapping, timeouts, reporting, and job-loop gating.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| tests/test_serverless/test_initializer.py | Adds unit tests for initializer execution, failure/timeout behavior, reporting, and gating before the job loop. |
| runpod/serverless/worker.py | Runs supervised initialization before starting the JobScaler/job loop. |
| runpod/serverless/modules/rp_initializer.py | Implements supervised initializer execution, timeout/error wrapping, structured logging, and best-effort reporting. |
| runpod/serverless/init.py | Documents new initializer and init_timeout config options in the serverless start() docstring. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
bugbot run |
- detect functools.partial and async __call__ initializers (were run on the sync path and never awaited, silently skipping init) - catch Exception, not BaseException, so KeyboardInterrupt/SystemExit from user code propagate instead of being reported as init failures - use a single unittest import style (CodeQL)
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 8 changed files in this pull request and generated no new comments.
Suppressed comments (4)
runpod/serverless/modules/rp_job.py:347
run_job_generatorbounds the handler/traceback string viaclip(...), but then appends up to 16KB of captured logs afterward, which can defeat the intended bounding and inflate the streamed error payload. Consider clipping the final assembled error string instead.
error = clip(f"handler: {str(err)} \ntraceback: {traceback.format_exc()}")
if captured_logs:
error += f"\nlogs:\n{captured_logs}"
yield {"error": error}
runpod/serverless/modules/rp_initializer.py:96
- Docstring typo: “abandoned an die” should be “abandoned and die”.
"""Run a blocking callable on a daemon thread instead of `asyncio.to_thread` so if stuck,
it can be abandoned an die without blocking the executor shutdown and process exit."""
runpod/serverless/modules/rp_initializer.py:140
run_initializer_asynctreatstimeout=0as “no timeout” because it uses a truthiness check (if timeout:). If a user setsinit_timeout: 0expecting an immediate timeout, it will instead run unbounded.
if timeout:
await asyncio.wait_for(asyncio.ensure_future(awaitable), timeout=timeout)
else:
await awaitable
runpod/serverless/init.py:150
- The docstring says the
initializerruns “before the job loop begins”, but the implementation can still acquire jobs while the initializer runs (the gate only prevents the handler from executing). This wording is misleading for SDK users.
config["initializer"] (Callable, optional): Startup work before the job loop begins,
e.g. loading a model or starting an inference engine.
A worker whose model load or engine start fails or hangs should report the error back to the platform.
This PR adds an optional
initializerandinit_timeoutto the serverless config. The SDK runs it before the worker takes any job. On failure or timeout it:init_failedlog.Testing